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INTRODUCTION 

The  main  focus  of  our  work  at  Rush  has  been  on  the  taping  of 
human- to-human  tutoring  sessions  and  the  analysis  of  the 
transcripts  of  those  sessions  with  regard  to  both  language  and 
content.  The  language  analysis  involves  study  of  vocabulary, 
syntax,  semantics,  and  discourse  for  use  in  the  understanding  of 
ill-formed  input  and  the  generation  of  responses.  The  analysis 
of  content  focuses  on  tutoring  rules,  modeling  rules,  and  the 
underlying  knowledge  used  by  the  tutor C 

At  IIT  we  are  working  on  the  development  of  the  lexical 
functional  grammar  rules  and  lexicon  for  the  cardiovascular 
sublanguage,  on  the  development  of  a  cardiovascular  knowledge 
base  for  the  tutoring  system,  on  a  Prolog  prototype  for  the 
tutor  we  are  trying  to  build,  and  on  the  initial  plans  for  a 
tutoring  system  in  Lisp  to  support  our  research  in  text 
generation.'' 

-  "y  jL  V 

The  context  of  our  work  is  an  attempt  to  build  an  ^intelligent"^ 

version  of  a  remarkably  effective  ^"conventional1^  tutoring 
program,  CIRCSIM,  designe£-jand  implemented.. in -Basic«jbY"JO'eT 
M-i chaei  anti  Alien  'Rovick-:  CIRCSIM  is  intended  to  ~tutor  first 
year  medical  students  in  the  complex  negative  feedback  processes 
used  by  the  human  body  to  maintain  blood  pressure.^  An  important 
first  stage  in  our  work  has  been  to  examine  and  mak^  explicit  the 
knowlege  base  and  tutoring  strategies  implicit  in  CI&CSIM  and  to 
study  the  ways  in  which  students  interact  with  this  program. 

Much  of  this  work  has  been  done  by  Michael  and  Rovick  at  Rush;  it 
has  been  used  by  an  IIT  Ph.D.  student  Nakhoon  Kim  in  'building  a 
prototype  version  of  the  "intelligent"  CIRCSIM-TUTOR .  - 

We  are  motivated  by  the  conviction  that  the  best  way  to  develop 
intelligent  tutoring  systems  is  by  evolution  not  revolution  - 
that  the  most  effective  ITS  will  evolve  from  the  most  effective 
conventional  systems.  The  insight  of  expert  teachers  is 
essential  -  as  is  observation  of  interactions  between  the 
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tutoring  system  and  real  students  of  the  material.  This  means 
that  it  is  necessary  to  maintain  a  working  system  at  each  step  of 
the  evolutionary  process.  Our  focus  is  on  the  provision  of 
natural  language  dialogue,  which  we  feel  is  a  key  ingredient  in 
tutoring.  The  use  of  Lexical  Functional  Grammar  reflects  our 
belief  that  it  combines  computational  tractibility  with 
psycholinguistic  insight  not  available  in  any  other  current  model 
of  language. 


ANALYSIS  OF  LIVE  TUTORING  SESSIONS 

In  May  1989  at  Rush  we  carried  an  experiment  in  which  we  asked 
four  medical  students  to  solve  a  particular  problem  in  cardiovascular 
physiology  with  the  assistance  of  an  experienced  tutor  (JAM  or 
AAR) .  The  problem  was  a  paper  and  pencil  exercise  requiring  the 
student  to  predict  the  qualitative  changes  that  will  occur  to 
seven  parameters  describing  a  negative  feedback  system  as  the 
result  of  applyting  a  perturbation  to  the  system.  The  particular 
negative  feedback  system  in  question  is  the  one  that  stabilizes 
blood  pressure  in  man.  The  tutor  attempted  to  assist  the  student 
in  arriving  at  a  correct  solution  to  the  problem  posed  and  a 
correct  understanding  of  the  domain  in  which  the  problem  is 
situated.  The  verbal  output  of  both  student  and  tutor  was 
recorded  and  transcribed  into  a  text  file  using  a  standard  word 
processing  program  (Word  Perfect  5.0). 

The  session  transcripts  are  being  studied  from  two  different 
points  of  view  -  we  need  to  perform  detailed  analyses  of  both  the 
language  and  the  content.  The  analysis  of  language  starts  off 
with  vocabulary,  syntax,  semantics,  and  discourse  structures. 

The  analysis  of  content  begins  with  a  search  for  tutoring  rules 
and  then  goes  on  to  student  modeling  rules.  We  are  also  studying 
the  underlying  domain  knowledge  used  by  the  tutors.  When  we 
have  worked  out  the  analysis  process  thoroughly,  we  plan  to 
record  a  number  of  sessions  in  which  our  colleagues  tutor  other 
students . 

The  first  step  in  analysis  is  a  simultaneous  review  of  tape  and 
transcript  by  one  of  the  tutors  to  correct  spelling  and  put  all 
sentences  and  sentence  fragments  Into  a  standard  form  with 
uniform  punctuation.  Then  a  simple  program  assigns  a  unique 
number  to  each  sentence  or  fragment  that  identifies  the  session, 
the  speaker  (tutor  or  student),  and  the  turn.  This  makes  it 
possible  to  analyze  each  sentence  separately  and  also  associate 
it  with  the  context  that  it  came  from.  We  already  have  programs 
to  pull  out  vocabulary  items  and  form  a  KWIC  index.  A  phrase 
counter  program  pulls  out  possibly  relevant  phrases  and  submits 
them  for  human  judgment.  We  are  planning  further  software  to  aid 
in  the  sublanguage  study.  So  far  our  attempts  to  extract 
tutoring  and  student  modeling  rules  from  the  sessions  have  been 
entirely  manual  -  a  kind  of  debriefing:  "What  were  you  trying  to 
do  here?"  "How  did  this  student  response  adjust  your  ideas  about 
where  the  confusion  lies?"  We  need  to  systematize  this  process 
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and  devise  a  better  method  of  representing  the  content. 


LEXICAL  FUNCTIONAL  GRAMMAR  ( LFG ) 

Ronald  Kaplan  of  Xerox  Palo  Alto  Research  Center  has  given  us  a 
copy  of  the  toolkit  for  Lexical  Functional  Grammar  that  he  has 
developed.  Ru-Charn  Chang,  an  IIT  graduate  student,  and  David 
Levin,  a  member  of  the  Philosophy  Department  of  the  University  of 
Wisconsin-Parkside ,  have  been  using  Kaplan's  system  in  developing 
grammar  rules  and  lexical  entries  for  the  cardiovascular 
sublanguage  used  in  Joel  Michael's  introductory  chapter  on 
cardiovascular  physiology.  Kaplan  is  interested  in  our  work 
because  we  are  one  of  the  first  groups  to  try  to  construct  a 
coherent  syntax  and  lexicon  describing  a  significant  subset  of 
English.  We  hope  that  this  part  of  our  research  will  be  of 
interest  in  itself  and  will  lead  to  separate  publications. 

The  Xerox  PARC  software  is  ideal  for  developing  grammar  rules  and 
vocabulary  for  the  cardiovascular  sublanguge,  but  it  responds 
much  too  slowly  for  use  in  a  tutoring  system.  Ru-Charn  Chang  has 
started  work  on  a  Prolog  prototype  of  an  LFG  parser.  It  uses  a 
bottom-up  left  corner  strategy  and  implements  some  but  not  all  of 
the  grammar  rules  from  the  collection  that  she  and  David  Levin 
have  put  together. 


DOMAIN  KNOWLEDGE  BASE 

A  domain  knowledge  base  Is  a  central  component  of  an  intelligent 
tutoring  system  and  the  contents  of  that  knowledge  base  are  as 
crucial  as  the  routines  that  search  it,  extract  knowledge  from 
it,  and  reason  with  that  knowledge.  We  expect  to  discover  more 
about  the  appropriate  contents  of  that  knowledge  base  and  how  it 
is  used  from  analysis  of  the  tv  coring  sessions.  We  are  also 
attacking  this  problem  using  tw  cher  approaches  -  one  based  on 
a  study  of  the  knowledge  embodiec.  in  CIRCSIM,  the  other  on  an 
analysis  of  a  manuscript  on  cardiovascular  physiology  written  by 
Joel  Michael. 

Glenn  Mayer  is  using  Naomi  Sager's  Linguistic  String  Parser  (LSP) 
to  analyze  this  manuscript.  He  has  begun  his  sublanguage 
analysis  by  extracting  words  and  phrases  and  building  a  KWIC 
index  of  the  manuscript.  (These  have  been  very  helpful  to  Chang 
and  Levin.)  He  is  now  adding  LSP  grammar  rules,  restrictions, 
and  vocabulary  classification  to  handle  sentences  that  either 
cannot  be  parsed  or  that  yield  inappropriate  parses.  His  plan  is 
to  convert  the  parse  trees  into  information  formats;  then  to 
convert  the  information  formats  into  frames  linked  together  by 
ISA  relationships  as  well  as  causal  and  spatial  relationships. 

It  is  clear  that  this  step  will  require  the  use  of  a  core 
knowledge  base  developed  by  hand,  corresponding  to  the  basic 
knowledge  that  Michael  expects  from  the  students  who  read  his 
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chapter  or  work  with  CIRCSIM.  We  have  had  a  most  convincing 
demonstration  of  the  need  for  this  knowledge  base  from  three  of 
the  Korean  students  involved  in  our  project.  They  could  not  read 
Michael's  chapter  in  spite  of  the  fact  that  they  can  speak 
English  fluently  and  can  read  complex  texts  in  computer  science. 
What  was  missing,  we  realized,  was  the  understanding  of  the 
western  view  of  how  the  human  body  functions,  which  most 
Americans  acquire  in  high  school  biology  courses,  if  not  earlier. 
Michael  gave  them  two  lectures  of  which  we  have  careful  notes. 

The  first  describes  in  elementary  terms  the  chambers  of  the 
heart,  the  main  arteries  and  veins,  and  the  lungs.  The  second 
describes  the  flow  of  blood  through  the  system  carrying  oxygen 
and  waste  products.  With  this  background  the  students  could  then 
read  Michael's  discussion  of  the  volume  and  pressure  mechanisms 
that  control  blood  pressure  -  the  core  material  for  CIRCSIM  and 
CIRCSIM-TUTOR. 

In  developing  the  knowledge  base  for  the  prototype  tutor 
described  below,  Nakhoon  Kim  translated  these  notes  into  Prolog 
facts  linked  together  in  a  semantic  net.  Yuemei  Zhang  has  now 
organized  this  information  into  a  collection  of  frames  for  the 
LISP  system  linked  together  by  a  number  of  different  semantic 
relationships.  She  has  also  developed  Lisp  routines  for 
retrieving  information  needed  by  her  text  generation  system. 


PROLOG  PROTOTYPE 

Nakhoon  KIm  has  developed  a  Prolog  prototype  for  CIRCSIM-TUTOR,  in 
partial  fulfillment  of  the  requirements  for  the  Ph.D.  in  Computer 
Science.  His  prototype  contains  a  screen  manager,  an  expert 
system  capable  of  solving  CIRCSIM  problems,  a  student  modeler,  a 
control  module  that  contains  lesson  planning  rules  and  simple 
tactical  rules,  and  a  simple  parser  for  student  questions.  The 
textual  response  to  these  questions  is  either  produced  by  filling 
in  the  blanks  or  totally  canned  (copied  from  the  CIRCSIM 
program) . 

The  Prolog  prototype  is  capable  of  carrying  through  a  CIRCSIM 
tutoring  session  with  a  student,  in  which  the  student  is  guided 
through  the  process  of  filling  out  a  prediction  table.  The 
system  detects  errrors  by  comparing  the  student's  solution  with 
its  own,  builds  a  simple  student  model  based  on  an  overlay  of  the 
concept  map,  comments  on  errors,  and  answers  simple  questions. 

The  development  of  the  prototype  has  given  us  the  opportunity  to 
work  out  simple  lesson  planning  rules  and  tactical  rules  to 
support  them.  It  has  forced  us  to  consider  the  student  modeling 
problem.  Most  important  of  all,  it  gives  us  a  setting  in  which 
to  collect  natural  language  input  from  students  and  watch  their 
reaction  to  natural  language  output. 

It  falls  short  of  the  CIRCSIM  TUTOR  that  we  plan  to  build  over 
the  next  two  years  in  four  important  ways: 
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(1)  The  control  module  now  handles  two  levels  of  rules,  lesson¬ 
planning  rules  and  tactical  rules.  We  need  the  capacity  to  do 
complex  multi-level  planning  of  the  tutoring  discourse.  The 
tutoring  rules  are  oversimplified  in  many  ways.  They  do  not 
teach  problem  solving  algorithms  at  all;  the  student  is  forced 
to  solve  subproblems  in  the  correct  order  by  the  program  flow. 

(2)  The  student  modeler  is  definitely  unsophisticated.  It  models 
the  student's  knowledge  of  physiology  processes  in  a  simple  one 
dimensional  fashion  and  does  not  even  try  to  model  the  problem 
solving  processes  involved. 

(3)  The  natural  language  umderstanding  system  can  understand  only 
simple  questions  from  the  student  phrased  in  complete,  perfectly 
grammatical  sentences. 

(4)  The  natural  language  generation  process  is  rudimentary. 


CIRCSIM-TUTOR  ON  THE  XEROX  LISP  MACHINE 

We  are  now  beginning  the  development  of  a  version  of  CIRCSIM- 
TUTOR  written  in  Common  Lisp.  Chong  Woo  is  working  on  a 
multilevel  planner  that  will  control  the  whole  system.  It  will 
interpret  rules  at  a  number  of  different  levels.  Leemseop  Shim 
is  writing  the  student  modeler,  trying  to  develop  a  system  that 
can  accommodate  both  overlay  and  bug  library  kinds  of 
information.  He  is  using  certainty  factors  to  express  the 
system's  level  of  confidence  in  the  student’s  knowledge  of 
particular  material.  Yoon  Hee  Lee  has  completed  a  spelling 
correction  program  and  is  working  on  an  LFG  parser  for  ill-formed 
input  in  LISP,  using  many  ideas  from  Ru-Charn  Chang's  Prolog 
parser  but  adding  a  chart  to  store  intermediate  results.  A  chart 
seems  the  most  natural  approach  to  handling  the  fragments  that 
appear  so  frequently  in  the  taped  sessions  and  that  we  expect 
will  be  even  more  frequent  when  the  students  are  using  the 
keyboard  for  natural  language  input . 

Yuemei  Zhang  is  starting  work  on  the  design  of  the  text 
generation  module.  As  mentioned  above  she  has  been  working  on 
the  problem  of  extracting  appropriate  material  from  the  knowledge 
base  to  support  text  generation.  She  is  also  studying  the 
discourse  structures  in  the  tutoring  sessions  to  obtain  ideas  for 
discourse  schemas  for  the  text  generator. 


OTHER  ACTIVITIES 

Four  of  us  went  to  the  International  Conference  on  Computer-Aided 
Learning  (ICCAL)  in  Dallas  in  May.  I  gave  an  invited  talk  on  the 
importance  of  natural  language  in  intelligent  tutoring  systems. 

I  also  took  part  in  a  panel  session  on  the  testing  of  tutors  in 
which  I  emphasized  the  importance  of  working  closely  with  experts 
in  the  teaching  of  the  domain  knowledge  and  on  trying  out  ideas 
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with  real  students.  Joel  Michael  and  Allen  Rovick  gave  papers 
about  the  pathophysiology  tutor  and  tutoring  rule  editor  designed 
and  implemented  by  IIT  student  M.  Moidul  Haque .  (We  had  expected 
Haque  to  give  one  of  these  papers  himself  but  he  was  detained  in 
India  by  his  wife's  illness.)  Nakhoon  Kim  presented  a  paper 
about  the  Prolog  prototype  and  gave  a  live  demonstration.  All 
three  of  these  papers  appear  in  the  Proceedings. 

Allen  Rovick  came  back  from  the  workshop  at  Cornell  full  of 
excitement  at  the  new  ideas  he  had  heard  in  discussions  there; 
we  have  all  been  reading  the  papers  he  brought  back. 

Since  Nakhoon  Kim's  thesis  work  is  an  essential  foundation  of  the 
research  supported  by  ONR,  we  used  ONR  funds  to  pay  for  his  trip 
to  Dallas  and  also  to  support  him  this  summer.  Yuemei  Zhang  has 
assumed  major  responsibility  for  the  text  generation  research  and 
therefore  we  supported  her  part  time  this  summer  and  we  are 
paying  both  tuition  and  stipend  for  her  this  fall. 

We  are  most  grateful  for  the  loan  of  a  XEROX  Lisp  machine  to 
Rush.  This  means  that  we  can  easily  demonstrate  com  onents  of 
the  Lisp  system  to  Michael  and  Rovick  while  they  are  still  in 
progress  and  also  that  students  will  be  able  to  use  successive 
versions  of  the  system  as  it  develops.  It  is  also  increasingly 
valuable  as  a  development  resource  as  more  of  us  become  involved 
in  programming  and  in  expanding  the  knowledge  base.  At  the  same 
time  we  are  looking  carefully  at  the  possibility  of  moving  our 
work  to  a  Macintosh.  We  are  moving  from  Interlisp  to  Common  Lisp 
as  quickly  as  possible  to  facilitate  a  change  of  hardware. 
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